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ABSTBACT 

Because of the need for a systea designed to 
continuously evaluate State and local programs of occupational 
education relative to the relevancy of educational training and 
accountability of fundsv the Illinois Three Phase Systea for 
statewide Evaluation of Occupational Prograas was foraulated to 
evaluate both prograa product and prograa process assessaent. Phase 
One of the systea begins with program planning decisions aade by 
local district personnel incorporating findings froa locally*directed 
evaluations. The results of this planning are reflected in the local 
District One and Five Tear Plan for Occupational Education which is 
prepared annually and sent to the Division of Vocational Education. 
Phase Two of the systea includes the annual monitoring and evaluation 
by the State of prograa changes and progress reflected in the 
district's plan. Phase Three provides for on'site visitation of local 
prograas to identify ezeaplary gualiiies or deficiencies in the 
prograa, and to aake specific recoaat^ndations for prograa 
iaproveaent. The docuaent includes a detailed description of Phase 
Three's seven stages: pre*visitation preparation, school personnel 
orientation, teaa orientation, teaa exploration, preparation of the 
evaluation report, suamary conference, and implementation of 
suggested solutions* Appended are the formats for the local plan and 
evaluation reports. (Author/BP) 
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OVliRVIEW 



The evaluation of American Educational processes and projects has received increased 
emphasis in recent years. During the past decade there has been more focus on evaluation of 
educational programs than in the previous fifty years combined. Moreover, the continuation of this 
trend can be expected. 

Tvw factors h«ve contributed greatly to the increased concern for evaluation: relevancy of 
educational training and accountability of funds. Evaluation of educational programs Is now a matter of public 
concern. The tax paying public has supported education for years and is now demanding to know the outcome 
of its investments. Occupational education has not been ignored in this specific vein. Evaluation is not only 
designed to show the public what has or has not been accomplished, but has a major role in Identifying 
strengths and weaknesses for educators at various levels to aid in Federal, State and local planning. 

Since the launching of Sputnik in 19B7, the need for persons with higher levels of t«Umical skills has 
been noted by our country's leaders. Enacted to supplement the Vocational Education Act of 1963. the 
Vocational Education Amendments of i;)68 represent the most recent plan for expanding and Improving 
proyams in occupational education to meet the Identified need. The mandate for state and local evaluation to 
determine whether the intentions of the legislature are being met was cleariy spelled out In these amendments. 
The Illinois Division of Vocational and Technical Education has established a system deslgnMl to accomplish 
the mandate for evaluation. 



THE ILLINOIS SYSTEM 



INTRODUCTION 

The State of Illinois is responsible for the annual distribution of ovar 30 mflHon dollars in State and 
Federal funds to help support over 8,(XX) specific occupational education programs. As an attempt to 1) insure 
optimum utilization of these State and Federal funds and 2) promote superior planning and operation of local 
programs, the Three Phase System for Statewide Evaluation of Owipational Education Programs has been 
developed. This system evolved through the cooperative efforts of local district personnel, individuals from 
higher education, business and industrial representatives, State Advisory CouncH members and State Office 
personnel. Several workshops, correspwdence with individuals, and a pilot test of the on-site phase of the 
system led to many stages of refinement. 

The Three Phase System incorporates the tasks of accountability, planning, and Improvenrrent of 
programs* The goals of the evaluation system are as follows; 

1. to promote and assist with the development of quality occupational education programs at 
the local level, 

2. to provide the Division of Vocational and Technical Education with data upon which better 
statewide planning of occupational education programs can be based, and 

3. to help assure accountability of Federal and State functe allocated to local occupational 
education programs. 

The initial step in establishing the above goals was the determination of the Goats for oc* 
cupational education in Illinois. Sugg^tions were obtained from individuals from througnout the 
State. These individuals were selected from groups of secondary, post secondary, university and 
business and industrial personnel. Once a bank of goals was identified, a subsequent request was 
made of individuals to rank the goals in order of priority. The riiSultant goals selected as first and 
second priority for the coming years are as follows: 

A. First Priority Goats 

1. Occupational prof^ams offered should be realistic In terms of current and projected 
manpower needs. 

2. The needs of all students (including target populations) desiring occupational 
training should be adequately met by occupational programs. 

3. The total education of students desiring occupational training should be an 
articulated effort from kindergarten throu^ adult education (occupational 
information, occupational orientation, and occupational training.) 

4. Adequate andilary services (guidance, placement, follow-up, student organizations, 
etc.) should be provided by local programs of occupational education. 

5. Occupational education should adequately equip students with job entry skills 
required to enter the world of work. 



ERLC 



^2- 



B, Second Priority Goals 



6. 

7. 
8. 
9. 



Program planning should be adaquate enough to ensure that occupational programs 
are appropriate. 

Adequate facilities should be available for the occupational programs being offered. 

Staff qualif iciation should be appropriate for occupational areas taught. 

Community resources should be utilized in planning and Implementing local 
programs of occupational education. 



The Three Phase System of Evaluation is designed to assess the extent to which the above goals are 
being attained. Each of the three phases has a specific function: 



PHASE ONE 

PHASE TWO 
PHASE THREE 



Utilization of locally-directed evaluation data in program 
planning by the local district culminating in the development of 
a One and Five Year Plan for Occupational Education. 

Program approval by staff of the Illinois Division of Vocational and 
Technical Education. 

External evaluation by a visiting team to develop a profile of the 
local program with conclusions, recommendations, and suggested 
solutions to the local district. 



Figure 1 presents the System in schematic form with a feedback loop between Phase Three and P'hase 
One. This feedback allows for a built-in follow-up of changes recommended by the team for the following 
year's Local Plan. Phases One and Two are annunl processes while the third phase occurs every two to five 
years or as warranted. 

" PHASE 1 




Figure 1 

Schematic of Three Pfwae System for 
Statewide Evaluation of Occupational Education Programs 



ERIC 



-3- 



PHasE onE 




Purpoift 

Phase One of the Three Phdse System for Stttewlde Evaluation of Oc(^pat!onal Education Programs 
is initiated by the staff of the local educational agency. It h this phase which incorporates the fir«dings of the 
district's locally-directed evaluation and the evaluation conducted by a visiting team (when available) in 
planning the district's occupational prof^ams. 

Prooadura 

The key to Phase One 1$ the planning conducted by local district personnel. Effective pjanning entails 
the use of a continuous evaluation system which has been designed to provide data on which program planning 
decisions can be based. 

A commitment to planning and evaluation by the local administration is essential. Planning must be a 
joint effort between local staff and community business and industry representatives (local advisory 
committee members) conducted under the direction of a good leadership team. It is essential that sufficient 
^ime be allowed to accomplish the planning tasks. A viable solution to the time problem has been the 
utilization of institute days authorized by the School Code to conduct organized locally*directed evaluation 
and planning processes. 

To a ^eat degree the success of Phase One is dependent upon the program evaluation and planning 
skills possesssed by local district personnel. The Division of Vocational and Technical Education has stressed 
the importance of improving these skills. The development and distribution of a locallyKifrected evaluation 
handbook and the offering of %vorkshops on evaluation and planning have been two such efforts. Both of these 
efforts stress the need for local evaluation of the product (students) and the processi^ (pro{^m operation). 
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Particular attention should be paid to product evaluation at the local level. This would include the 
measurement of student performance, follow-up of former studente, interviews with employers of former 
students and ratings of training received by former students. 

Most program processes to be evaluated by the local district correspond easily to the Areas of 
Concern for Evaluation identified during Phase Three of the state-wide system. Local evaluating teams 
consisting of students, teachers, administrators, employers, employees, local advisory committee members, etc. 
should be organized to examine each of these areas in detail. Students served, occupational programs, 
administrative organization, personnel, program objectives, evaluation processes, guidance services, and 
resources utilized are the eight areas which have been identified as essential to the occupational program. The 
information provided from the locally-directed evaluation would provide adequate background data upon 
which to base decisions regarding short-range and long-range program plans and priorities. 

Outcome 

The outcome of Phase One is a document which serves as a contractual agreement with the Division 
of Vocational and Techncial Education as well as ? planning aid to the local district. The Division has prepared 
a set of Guidelines and a Format for developing the Local Plan. The Plan contains eight sections: A) General 
Information, Bt Administrative Organization, C) District Objectives, Policies, and Program Description, D) 
Proposed One-year Plan, E) Long-range Plan, F) Financial, G) Summary, and H) Signature Page for the Chief 
School Administrator. A mora detailed description of the Local Plan may be viewed In Appendix A. 



PHaSE TWO 




Piirpom 

Phdsd Two of the evaluation system is devoted to the review and evaluation of the Local District One 
and Five Year Plan for Vocational and Technical Education by the staff of the Division's Program Approval 
and Evaluation Unit 

ftoeedure 

Once prepared, the Local Plan is reviewed and approved by the local governing beard and • 
transmitted through the Superintendent of Educational Service Region or the Illinois Community 
College Board to the Division of Vocational and Technical Education. The staff of the Division's 
Program Approval and Evaluation Unit, with assistance from other state staff members. Is responsible 
for evaluating the Local Plan and making recommendations to the State Director of Vocational and 
Technical Education regarding approval of tha district's occupational programs. 



Although standards for all sdiools era not establishad, a certain degree of quality Is louiiht in the 
nan-ative part (Section C) of the Local Plan. Each school must be considered separately as it exists within its 
own commu;iity* Such things are p^ performance, population, finar^cial ^pport and other existing training 
opportunities must be taken into account. If portions of this section of ti>a Local Plan are not clearly 
explained, the local district is asked to provide further information* Possibly of graater concern during Ptuse II 
is the amassment of the local educational agency's progress and improvements shown from or^ year to tfie 
next 

The Local Plan also provides a form on which to report occupational education programs offered by 
the local district T^te staff of the Program Approval and Evaluation Unit make approval recommendations to 
the State Director regarding each of tf^ courses offered. If sufficient information is not available to make a 
decision regarding specific programs or courses, the Unit may request more detailed course ctescrfptions from 
the local district. 
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Outcome 



nir«.t«r n " ^" 9'^^" ^0 occupational course by the State 

Director as well as an overall approval status of the Local Plan which may fall in one of the following 
categories: Approved. Conditionally Approved, or Not Approved. Tonowing 
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PHaSE THrEE 




Phase Three Is designed to provide a local district with a profile of Its total OKupational program 
through the utilization of a visitation team comprising external business and industrial representative 
(management, tabor and others), occupational educators and students from outside the district. Tnfs profile 
can be compared to the profile developed by the local district personnel in Phase One (Local District Pl4in). 
The results of this phase provide data to direct the improvement of total occupational education program 
planning in the local district and in the State, Phase III also ensures the accountability of funds allocated to 
each district. 

Procedure 

This phase is accomplished through on-site visitation by a team of individuals not associated with the 
district The team has the responsibllty of gathering and analyzing data concerning the district's total 
occupational program. These data are collected through analysis of the Local District Plan; analysis of certain 
school and community data provided by the local school; analysis of a preliminary evaluation questionnaire 
administered to administrators, teachers, counselors, and students; interviews with the above mentioned 
groups and business and industrial representatives of the community: and gent^.-al observations. 

All of the data collected are compiled to make conclusions, recommendations and suggested solutions 
with regard to eight Areas of Concern for Evaluation relating to total occupational programs. The eight areas 
and the related goal of each are as follows: 
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CONCERN 



PRIMARY GOAL 



Students Served Determine the degree to which st jdent needs, interests and abilities are 

being met through the occupational programs offered by the district. 

Occupational Programs Determine the scope and effectiveness of the district's occupational 

programs. 

Administrative Organization Determine the structure and effectiveness of the oa:upaiio«ial administrative 

organization. 

Determine the qualifications, working relationships and professional 
development of the occupational administrative, instructional and 
guidance personnel. 

Determine if objectives are developed in measurable terms and utilized in all 
levels of instruction. 

Determine the effectiveness of the locally directed evaluation system. 

Determine the extent and effectiveness of the utilization of external and 
internal resources. 

Determine the quality and quantity of guidance services provided to 
occupational students. 

These Areas of Concern for Evaluation provide the primary focus of the visitation. The length of the 
visitation will vary with schools of extreme size (very large or very small), but the average visitation will 
require two days in the district for interviews and observations by the team end part of a third day for a 
Summary Conference to present the team's findings to the local district administration. 

The procedures of the visitation can be broken Into seven specific stages: 

1. Pre-visitation Preparation 

The district to be visited is notified of the visitation dates and the names of visitation team 
members. A request for specific school and community data and a copy of the One and Five 
Year Plan to be sent to team members is also made* 

2« Orientation of School Personnel 

A slide-tape presentation regarding the purpc^se, (tescription and schedule of events foi Phase 
II Ms shown by a state staff member to the local district occupational education personnel. 
Preliminary Evaluation Instruments designed to gather specific evaluative data relating to the 
district's total occupational program are administered to the staff and a sample of students 
prior to the on-site visitation. 

3. Orientation of Team Members 

The philosophy of the visitation, the mechanics involved, and tne review and discussion of 
available information is covered with the visitation team by the team leader. 



Personnel 

Objectives 

Evaluation 
Resources Utilized 
Guidance Services 
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4. Team Exploration 

The visitation conduots Interviews with local liistrict staff, students, and community 
repre^ntativds. 

5. Final Report Preparation 

A composite of team observations is compiled m6 available data synthesized; these are 
organized in a report format and present&d to the district. This report contains both positive 
and negative conclusions, along with recommendations ar^d suggested solutions for each total 
program Area of Concern. 

6. Sumnr>ary Conference 

The team leader presents the visitation team's conclusions, recommendations and 
sugge^ited solutions for each total program area to the local district admintstrators. At 
this meeting, attended by the Division's Regional Vocational Director, the feasibility of 
fuiftlling the recommendations presented is determined. 

7. Implementation of Recommendations 

The local district implements the appropriate action to remedy the deficiencies identified by 
the visitation team. 

Outcome 

The outcome of Phase Three is a written evaluation report. This r^rt stresses recommendations and 
suggested solutions which are c^mnionsurate to the specific district's ability to adiieve them. The report is 
duplicated and sent to tiie local chief school administrator for distribution to the local staff and discu&sion* 
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SUMMARY 



The need for a system designed to continuously evaluate local district total programs of occupational 
education was most apparent to the Illinois Division of Vocational and Technical Education. It was essential 
that the evaluation system include both program product and program process assessment. To meet these 
needs, the The Three Phase System for Statewide Evaluation of Occupational Programs was formulated. 

Phase One of the System begins with program planning decisions made by local district 
personnel based on results from the conduct of locally-directed evaluation. The results of this planning 
are reflected in the Local District One and Five Year Plan for Occupational Education which is prepared 
annually and sent to the Division of Vocational and Technical Education. Phase Two of the System 
includes the annual monitoring of program changes and progress reflected in the district's plan by 
state staff personnel. Phase Three provides for on-site visitation of local programs by a visitation team 
consisting of educators, business and industrial representatives and recent students of occupational 
programs. Phase Three results in the identification of exemplary qualities and deficiencies in the 
occupational program, with specific recommendations for program improvement where needed. 

The interlinkage of Phase Three to Phase One is the essential aspect of the Evaluative System. The 
Local Plan prepared following the on-site visit provides Immediate follow-up of the local district progress 
toward the remedy of occupational program deficiencies identified during Phase Three, 
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DETAILED DESCRIPTION OF PHASE THREE 

STAGE 1 
PRE-VISITATION PREPARATION 

IDENTIFICATION OF SCHOOLS 

Schools to be visited are Identified by the Illinois Division of Vocational and Technical Education. 
Selections are nnade by a sampling procedure deigned to identify a representative sample of the schools within 
the state so that each year a number of ^ondary schools, yrea vocational centers, and po%t-secondary 
institutions are visited. 

NOTIFICATION OF VISITATION 

Each district to be visited is notified by the Superintendent of Educational Service Region or 
the Community College Bc^rd during the spring of the year preceding the visitation. After this 
notification, materials explaining the visitation in more detail are sent to the chief school administrator. 

Materials mailed to each district scheduled for a visitation include the following: 

1 . A statement of visitation purpose 

2. Notification of exact dates for the visitation. 
3* A schedule of events for the vl$Jtation 

4. Notification that later contact will be made to establish a date Vor the orientation of local 
district personnel involved with the occupational program. 

The chief school administrator is requested at the time of notification to mail the following 
information to each team member 3^ weeeks prior to the visitation date« 

K Local District One and Five-Year Plan for Vocational and Technical Education 

2, School and Community Data Form Which includes: 



a. 


Occupational program enrolln>ent lunduplicated) 


b. 


Advisory committea information 


c. 


Area unemployment ra^ 


d. 


Description of local follow-up procedures 


e. 


Description of placttmant acth/ities 



VISITATION TEAM 

The staff of the Divisicn's Program Approval and Evaluation Unit is responsible for selecting and 
appointing qualified team leaders and team members to school visitation teams. All team personnel are 
aalected from outside the district to be visited. 

The number of teachers employed in an occuational program is determined by the size of the 
visitation team. A tentative (^Ide to the number of team members is as follows; 
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Number of Occupational Teachers 



Number of Team Members 



less than 5 -2- 
5-10 -3- 

16 ^ 20 .5- 

2H .5+ 1 for each 

additional 5 teachers 



The team will consist of a team leader and members representing these three specific groups: 
1. Educ^jtors 



2. Business and Industry personnel 

3. Fornrter occupational students 



Team Leacier. The team leader will hold the primary responsibility for organizing the visitation. His or 
her duties will include orienting the team, scheduling interviews, coordinating the visitation, preparing the final 
report with the aid of i:he team, and presenting the visitation findings to the local district persortnel. 



The fotlowinfi criteria are used in the selection of team leaders: 



1 • Experience or background in occupational education, 
2. Familiarity with the needs of various size schools, 
3* Competence as preceived by colleagues, and 



4. Availability to serve. 



Team leaders participate in an orientation workshop, sponsored by the Division, which is designed to 
prepare them to conduct several visitations durng the school year. Such an activity is conducted in an effort 
to insure the uniformity of visitations. The final rt^quirem^t of team leaders is participation In a workshop to 
be held at the end of each evaluation year. Thfs workshop has two basic functions: the summarization of the 
year's findings and the ref inentent of the Evaluation System for use in subsequent years. 
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Educators. The educators to serve on the visitation team are selected by the Division's 
Pfuyram Approval and Evaluatiofi Unit staff. Suggested names and addresses are annually ruceived 
Stom the chief school administratorb of districts having approved occupational programs. When dates 
are finalized for each visitation, individuals are selected from the bank of names and letters are sent to 
request thoii services for that visitation. 

Several criteria are used in both the request for names and In the actual selection of educators for 
team members. The criteria are: 

1. Knowledge of occupational programs, 

2. Teaching or administrative expertise in occuational education, 

3. Recommendations by colleagues, and 

4. Availabilty to sen^e. 

Information regarding the visitation procedures is mailed to team members. This is supplemented by 
an orientation session conducted by the team leader upon the group's arrival at the visitation site. 

Orienting the team to the art of interviewing and to efficient use of available data is a responsibility 
of the team leader. The educators on the team spend the majority (70%) of their exploration time interviewing 
educators within the school, and the remainder interviewing students (20%) and surveying busing and 
industrial personnel of the community (10%). 

Reprasentativii of Business and Industry. Representatives of the business and industrial community 
are selected by the staff of the Division of Vocational and Technical Education. The source of business and 
industrial representatives is established in the same manner as the bank of educators; a request made to chief 
school administrators. 

Criteria for the selection of business and industrial representative are listed below: 

1 . Concern for and basic understanding of the goals of occupational education, 

2. Recommendation by colleagues, and 

3. Availability. 

Business and industrial representatives^ like other team members, are selected several months prior to 
a particular visitation, and invitations to serve are mailed to each. Additional information regarding the 
visitation is mailed to business and industrial representatives and is supplemented by the team leader during 
the orientation. 

The business and industrial representatives spend approximately 70% of their time in the community 
talking with employers and other people who have contact with current students, former students, or have 
direct contact with the school. The remainder of their exploration t?me is devoted to interviewing and 
observing occupational educators within the school (20%) and interviewing students (10%). 
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Former Students. Former students of occupational education programs from outside the district are 
selected to serve as team members. These team members are able to gather pertinent information from 
presently enrolled studente which may be inaccessible to educators or business and industrial representatives. 

Selection of former students is made by the evaluation team leade . The Division staff asks that the 
following criteria be utilized in selection: 

1. Ability to relate to both students and adults, 

2. Enrollment in a secondary or post^secondary occupational program within the past three 
ytars, and 

3« Availability to serve. 

Each former student team member is paired with a student currently enrolled in the occupational 
program who has been released from classes for the day or for a portion of the day, The currently enrolled 
student acts as a guide and a3 a facilitator for the former student's interviews with other students within the 
school The former student spends approximately 80% of his or her time interviewing students while the 
remainder is spent with teachers and counselor, 

Formet student team members also receive a mailing of information regarding the evaluation system 
and a description of their role within the system. 
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STAGE 2 

SCHOOL PERSONNEL ORIENTATION 



Approximately one month prior to each visitation an orientation presentation is made to the local 
district personnel. This presentation is given bv a staff member of the Division of Vocational and Technical 
Education at a location designated by the chief school administrator from the district to be visited. 

To ensure uniformltv among presentations, and to consume a minimum of staff time, all such 
presentations are maue through a slide-tape medium. The actual presentation takes approximately thirty 
minutes. » 

The presentation stresses certain aspects of the Three Phase System for Evaluation, and explains the 
purpose of the visitation, the structure of the visitation team, the mechanics of data collection, the eight areas 
of concern for evaluation, the format for the final reports, and the Summary Conference. Each of the three 
phases is described and examined. School personnel are also given a brochure summarizing the presentation for 
future reference. Upon completion of the audio*visual presentation, the state staff member entertains any 
questions from ttie staff regarding the Evaluation System. 

A second major task completed at the time of school personnel orientation is the administration of 
Preliminary Evaluation Instruments to all occupational education instructional and prof^lonal supportive 
staff. These instruments have been designed to collect preliminary data regarding the local occupational 
education program to aid both the team and the State Division. The Preliminary Evaluation Instruments are 
administered to all staff who are involved with occupational programs and to a sample of students enrolled in 
occupational courses. The responses to these instruments are sent to the State Office for tabulation and 
summarization. Results are distributed to the team visiting that particular school prior to their visit. 
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STAGES 
TEAM ORIENTATION 



Orientation of team members begins with the nailing of certain informaticm to each member prior to 
;he visitation. A packet of materials is mailed to each team member by the local district. This packet contains: 

1- The Local District One and Five Year Plan for Occupational Education, and 

2. Completed School and Community Data Form. 
Materials mailed to each team member from tte Division Includes: 

1. The results of the iVeliminary Evaluation Instruments, 

2. A publication describing the philosophy and mechanics of the evaluation, 

3. A schedule of events for the specific evaluation at hand, 
^* Team member handbook, and 

5. A summary of follow-up results for part of the former student body. 

This packet gives the team information regarding the Evaluation System prior to the visitation as well as 
specific information about the local school to be visited. 

The team leader has the responsiblity of conducting the team orientation. Orientation involves three 
major areas of discuriion: the philosophy of evaluation, the mechanics of the visitation, and a review and 
discussion of available information. 

A discussion of the philosophy of evaluation is led by the team leader. A slide-tape presentation 
which has been specifically designed to orient team members is utilized to present general information and to 
stimulate discussion. 

Most of the orientation time is spent discussing the mechanics of the visitation. The primary 
goal is to make sure each team member understands his or her role and is prepared to fulfill that role. 
The team leader stresses the eight Areas of Concern for Evaluation and makes some basic suggestions 
concerning the assessment of these areas through interviews and observation. The team leader also 
verifies the time and place of interviews for each team member. The format for the final report is 
reviewed to guide team members in the writing of their comments which result from interviews and 
observations. 

The remainder of the orientaiion meeting is spent discussing the information which is already 
available from the local district and the Division. At this time, the team discusses important points resulting 
from the review of the Local District Plan and results of the Preliminary Evaluation Instruments. At th/e same 
time, the team members discuss specific concerns which they have formulated through a review of the 
information packet received prior to the orientation meeting. 
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STAGE 4 
TEAM EXPLORATION 



Stage four involves the on-site visitation of the local district by the team to gather additional data 
with which to assess the program. Each of the eight Areas of Concern for Evaluation-students served, 
occupational programs, administrative organization, personnel, objectives, evaluation, ^idance services, and 
resources utiltzed-are explored in detail. Each team member is charged with interviewing individuals and 
drawing conclusions for each of the Eight Areas of Concern for Evaluation from the interviews. Individuals 
interviewing educators within the school cover all eight areas in their interviews, while the former student and 
business and industrial team members concentrate on areas related to the personnel which they interview. 

Stage four begins on the morning of the first day following the evening orientation. Prior to the start 
of school on that morning, a meeting of the visiiation team with the school administration and occupational 
education instructional staff is held. The meeting, which lasts approximately thirty minutes, Involves two 
components: ttie introduction of the visitation team to the staff and a presentation given to the team by the 
chief school administrator or his designee. 

The following questions act as a guide for the presentation to the team: 

1. What occupational programs are currently offered and what part do they play in the 
district's total education of students? 

2. What major changes have been made in occupational programs during the last two years? 

3. Wha^ are the projections for future occupational programs in the district, including 
maintenance of programs, extension of programs, improvement of programs, and new 
program projectives? 

4. What are the limitations facing the occupational programs which are classified as internal 
(physical and financial) and external (community and state)? 

5. How does the district currently evaluate the ongoing occupational education program? 

Following the presentation by the chief school administrator, the staff is dismissed to begin 
the school day while the visitation team spends an additional twenty to thirty minutes asking questions 
or discussing matters of concern with the administrator. 

Upon completion of the discussion with the administrator, the team meets for a brief coordination 
meeting. The team leader reviews the duties of the team members with them and checks to see that each 
member clearly understands the taiks of the day. 

Individual interviews begin following this meeting (approximately at the beginning of the second 
schpol period). A special type of interview schedule (Team Member Handbook) is used which gives the 
primary goal and interview tasks, and sources of data for each of the eight areas of concern (see Figure 2 for an 
example). 

The interviewer works from a group of tasks in each area of concern for evaluation to which he must 
respond at the conclusion of the interview. This procedure requires the interviewer to formulate his own 
specific questions to complete the interview tasks. 
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FIGURE 2 

Example of Recording Fonn for One Area of Concern for 

Evaluation 



Specific tasks are given below the Primary Goal under the heading of "Interview Tasl<s " 
Following the Interview Tasks on the form is the section entitled "Data from Other Sources." This 
section includes answers to questions or data which have been gathered through other means, i e 
Preliminary Evaluation Instrument results, local plans. School and Community Data, etc. During the 
team orientation, the team leader goes through each of the eight areas with the team, filling in the 
data from other sources. 

Upon completion of each interview, the interviewer is given time to record conclusions regarding each 
of the areas considered on the recording form. 

Individuals who are interviewed fall into one of tfw following categories: 
1. Teachers 

a. all occupational education instructors 

b. a sample of academic instructors 
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2. 



Administrators 



3. 



a. Board of Education members 

b. Chief school administrator 

c. Curriculum director 
d: Business manager 

e. Vocational director 

f . Building principals or deans 
3. Department chairmen 

h. Other administrative personnel 
Guidance personnel 



a. 
b. 



Director and/or chairman of guidance personnel 
Guidance Personnel 



4. 



Learning resource personnel 



5. 



Business and Industrial Personnel 



a. 
b. 



Advisory committee ntembers 

Representative of community agencies such as : Illinois State Employment Service, 



c. 



Chamber of Commerce, etc. 
Employers of former students 



6. 



Students 



a sample of ail students. 



interviews with faculty members within the school are held on an individual basis or conducted in 
small groups. Business and industrial memt>ers of the community can be interviewed most effectively through 
small group meetings involving three or four people. In terms of the amount of time required of the 
community people, this type of interview has proven most efficient. Formal interviews with students are also 
conducted in small groups. Students appear to be more open in their comments in small groups as compared to 
individual interviews. Althou^, individual contact with students is made by team members on an informal 
basis throughout the day. 

The length of interviews within the school is somewhat governed by the length of a given 
class period. However, in most cases the interview should last for at least thirty to forty minutes; arid 
additional twenty minutes should be available to the interviewer for recording comments. Student 
inte 'views are held on approximately a thirty .'ninute schedule while the interviews with community 
reprssentatives require approximately one hour. 
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STAGE B 

PREPARATION OF THE EVALUATION REPORT 



The Evaluation Report is a document prepared bv the team to reflect their assessment of the local 
occupational education program in totah The report is organized to correspond to the primary focus of the 
evaluation-the Eight Areas of Concern for Evaluation, A separate section of the nsport is devoted to each of 
the eight areas and includes conclusions, recommendations and suggested solutions for each. Figure 3 gives an 
example of one section of a final report. 
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FIGURE 3 

Example of One Final Repmt ArM of Con^m for 

Evaluation 



The report is prepared throu^ a cooperative effort involving all team members, with the team leader 
acting as discussion leader and recorder. It is tl^ team leader's responsibilty to place the report in proper order 
and format. 



On the evening of the first day of interviewing, the team meets to discuss the results of the day's 
interviews. An effort is made at this meeting to begin forming conclusions for the final report. Areas that 
require further search before definite conclusions can be reached are also identified at this time. The following 
morning is spent interviewing more individuals with the intent of supplementing those areas deficient in data. 
The team leader spends a portion of that morning writing up the results of the previous evening's discussion. 
This allows the team to make additions and corrections to statements during the afternoon report-writing 
session. This time schedule allows each team member to contrilnite to the report and review it before leaving 
the visitation site. 



21- 



STAGE 6 
SUMMARY CONFERENCE 



A Summary Conference is held upon completion of the evaluation report for the purpose of 
communicating the team's findings to the administrators of the local educational agency. This meeting 
provides an opportunity for the administrators to discuss the findings and to make l<nown any il-niiations the 
district may have in implementing the recommended changes. 

Three groups of individuals are represented at the Summary Conference: the visitation team, the local 
district, and tha Illinois Division of Vocational and Technical Education. The team leader represents the team, 
the chief school administrator represents the local district, and the Regional Vocational Director represents the 
Division. 

A list of additional individuals to represent the local district include: 



1. 


Superintendent of Educational Service Region, 


2. 


Board Members, 


3. 


Administrative staff. 


4. 


AdviMry committee members. 


5. 


Local Plan author, and 


6. 


Representation of the instructional 



The summary group should be of a size which can be accommodated in a small conference i oom to promote 
more effective discussion. 

The Regional Vocational Director serves as the moderator of the Summary Conference. It is his 
responsiWity to reinforce the objectives of the meeting and to coordinate the meeting in terms of those 
objectives. The team leader presents the findings of the team to the local personnel. This includes presentation 
and discussion of each area of concern separately. Major emphasis is placed on areas which can be improved, as 
compared with "exemplary" areas. 

The meeting does not exceed three hours and it is the responsibllty of the Regional Vocational 
Director to insure that sufficient progress is made to cover all necesary Information within the time period. 

The team leader and the Regional Vocational Director make any necessary changes or corrections in 
the evaluation report. The report is then transmitted to the State Division through the Regional Vocational 
Director for typing and reproduction. The State Division malls the evaluation reports to the chief school 
administrator for distribution to the staff, board members, and advisory committee members. Copies are also 
distributed to team members, the Regional Director, the State Advisory Council, and the Division Director's 
Office, and e copy is placed in the Division's Local Plan file. 

It is the responsibllty of the chief school administrator to present the Evaluation Report to the staff, 
board, and advisory committee for discussion. 



STAGE 7 

IMPLEMENTATION OF REPORT RECOMMENDATIONS 

Stage seven is a very important link in the entire Three Phase System. Without this stage, the effort 
expended during the other stage* is futile. This stage laegins with a meeting within the local district held by the 
chief school admmistrator or hi& designee for the purpose of discussing the findings of the visitation team. 

Consultant help should be sought from the Division of Vocational and Technical Education or from 
other sources when needed. These resource personnel should be utilized in the planning and actual 
implementation of changes in the local programs. 

This stage is tied directly to Phase One of the Three Phase System which has the primary function of 
planning. The on-site evaluation, or Phase Three, is only the beginning of the improvement of programs. It is 
necessary that the local school continuously evaluate its own programs to aid in future planning. 

Division staff review local plans and make follow-up visits to evaluated schools during Phase Two to 
determine the extent to which recommendations have been acted upon. The app. jved status of local plans 
submitted to the Division is dependent, in part, upon the action taken by a local district regarding the 
evaluation recommendations. 
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The local plan consists of eight specific sections which are identified by the letters A through H. Each 
section is identified and briefly described betow. 

Section A - General Information. Section A consists of certain vital or demographic information 
related to the local school district and community. Included are items such as legal name of school, name of 
Local Chief School Administrator, size and tax information of the district, average expenditures of district, 
and estimated student population of the school. 

Section B - Administrative Organization. Section B includes a description of the ad- 
ministrative structure and organization of the Occupational and /or Technical Division of the local 
district, as well as the organizational structure of any joint agreements in which the district is par< 
ticipating. This organization is described in a brief narrative with a supplemental chart, if necessary. In 
addition, specific questions are asked regarding the organizational structure. 

Section C District Occupational Educations Program Description. Section C in- 
volves a narrative description of how the local plan was developed, the needs that it attempts to fill, 
the students that are to be reached, the local plan for evaluating the program and plans for the future 
to improve both the number of offerings and the quality of occupational education in the district. Eight 
specific questions must be answered by the local school personnel in this section. They include: 

1 . What are the OBJECTIVES of the district's total occupational program? 

State the objectives, in measurable terms, as they apply to the district's proposed total 
occupational education program. 

2. How will proposed district's occupational objectives meet the needs of all students? 

a. A brief, concise course description with credits or contact hours granted, related 
instruction and on4he*job coordination for cooperative education program, must 
be included for each course listed in Section D. A description of the elementary 
occupational information program must be submitted if applicable. 

b. List the criteria used in identifying disadvantaged and handicapped students and the 
additional services provided to these students that would justify the added funding 
factor. 

c. List programs you propose to operate under special contract. 

3. What and how were school and community resources used in developing your program? 

a. Advisory Councils: 

1. Type 

2. Occupational representation 

3. Utilization 

b. Identify other community as well as school resources that were used in the 
development of your program. 

c. What resources were used in the planning of your program objectives? 

d. How will llu. school's progiam be articulated with various levels of education: i.e., 
K12, Post Secondary, |)rivatc schools? 
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4. What are the ancillary and guidance services being provided to students In the occupational 
piogram? 

a. Specific guidances services 

b. Placement services 

c. ^ Follow-up services 

d. Student organizations and/or activities provided for occupational students. 

5. How do you plan to evaluate the occupational program? 

Describe the components to be used in a formal, locally directed evaluation of the 
occupational program, taking Into consideration the relationship of the totnl program 
objectives to the results of the formal locally directed evaluation, student and community 
needs survey, placement information mni follow-up studies. 

6. Personnel qualifications in relationship to occupational program asslf^ment 

Describe the actions that are being taken to remove the deficiency of staff members not 
meeting the minimum requirements of the DVTE Bulletin No. 4. 

7. What program and/or curriculum improvments are proposed for the coming year? 

a. What curriculum modifications will be emphasized to improve the school's 
programs? 

b. Those districts receiving less than full approval on their previous Plan must dm:ribe 
progress towarcj meeting the criteria established for full approval. 

8. Describe the activities proposed to improve program quality within five years? 

a. Staffing and planned fn-$ervlc» training of staff 

b. Physical facilities and equipment 

c. Cooperative arrangement with other districts 

d. Use of area resources 



Section D - Proposed One-Year Plan. Section D consists of two major parts-the first 
part records all credit programs taught in high schools and community colleges. The second part 
reports elementary programs and/or non-credit programs (often called adult programs) offered by 
high schools and community colleges. 

Section F - FinanciaL Section F includes a tentative budget, indicating the sources of 
income and the major categories of expenditures for the proposed program. Separate spaces are 
provided for different types of districts and area vocational centers. An estimate of per credit cost is 
also requested. 

Section G - Summary. Section G summarizes the enrollments at various educational levels 
for three years along with a listing of occupational administrators, instructors, and guidance coor- 
dinators, The actual enrollments for the last school year, the actual enrollments for the current year, 
and the projected enrollment for the next school year are included. Head count enrollment figures 
(unduplicated} will be used. 

Section H - Signature Page. Section H provides a record of signature for a Board Officer, 
Local Chief School Officer, and the Superintendent of Educational Service Region. 
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Title Page. The title page includes the legal name of the school district and the dates of the visitation. 
Team Listing* This page contains the names and titles of the visitation team. 

Introduction. This section gives a statement of intent of the local evaluation by the Division of 
Vocational and Technical Education < 

General Conclusions and Determinations. This section is devoted to a narrative description of the 
exemplary qualities of the occupational education program and the major deficiencies that were determined. 

Conclusions, Recommendations^ and Suggested Solutions for Each Area of Concern for Evaluation. 

This portion of the Evaluation Report is divided into the eight Areas of Concern for Evaluation. Within each 
area three categories of information are provided: Conclusions, Recommendations and Suggested Solutions. 
Conclusions may indicate either exemplary or deficient program characteristics. Recommendations and 
Suggested Solutions are provided for only the Conclusions related to deficiencies. 
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